Decomposition Methods for Solving Finite-Horizon Large MDPs

Conventional algorithms for solving Markov decision processes (MDPs) become intractable a large finite state and action spaces. Several studies have been devoted to this issue, but most of them only treat infinite-horizon MDPs. This paper is one the first works deal with non-stationary finite-horizon MDPs by proposing new decomposition approach, which consists in partitioning problem into smaller restricted MDPs, each MDP solved independently, specific order, using proposed hierarchical backward induction (HBI) algorithm based on (BI) algorithm. Next, sub-local solutions are combined obtain global solution. An example racetrack problems shows performance proposal technique.